Semantic Question-Answering with Video and Eye-Tracking Data: AI Foundations for Human Visual Perception Driven Cognitive Film Studies
نویسندگان
چکیده
We present a computational framework for the grounding and semantic interpretation of dynamic visuo-spatial imagery consisting of video and eyetracking data. Driven by cognitive film studies and visual perception research, we demonstrate key technological capabilities aimed at investigating attention & recipient effects vis-a-vis the motion picture; this encompasses high-level analysis of subject’s visual fixation patterns and correlating this with (deep) semantic analysis of the dynamic visual data (e.g., fixation on movie characters, influence of cinematographic devices such as cuts). The framework and its application as a general AI-based assistive technology platform —integrating vision & KR— for cognitive film studies is highlighted.
منابع مشابه
Eye-Tracking Method’ Usage for Understanding the Cognitive Processes in Multimedia Learning
Introduction: Designing multimedia learning environments should consist of the evidence-based study and principals about the human learning process. Eye tracking is a way based on the learner processing of learning materials which presented in multimedia learning environments. The aim of the study was to examine the use of the eye-tracking method to investigate the cognitive processes in m...
متن کاملChronic Pain and Selective Attention to Pain Arousing Daily Activity Pictures: Evidence From an Eye Tracking Study
Introduction: According to the pain research literature, attentional bias for pain is the mechanism responsible for the development and maintenance of fear of pain in patients with chronic pain. However, there is still some debate about the exact mechanism and the role of faster engagement versus difficulty in disengagement in the development of attentional bias. Methods: To investigate ...
متن کاملQuest: a Model of Question Answering
We have developed a computer model of human question answering, called QUEST. QUEST simulates tile answers that people produce when they answer different types of questions, such as why, how, when, where, what-if, and yes/no verification questions. When QUEST answers a particular question, the model identifies relevant information sources and taps information within each source. Each informatio...
متن کاملApplication of eye tracking technology for screening and rehabilitation of children with special needs.
Eye tracking is a technology for measuring eye movements when looking at a point. The length of time to look at a particular situation, the sequence of observations, as well as points of attention are processed in the process of eye tracking and eye movements are identified by topics such as fixation, saccade, etc. Information about how the eyes move at a special moment has many potential appli...
متن کاملP58: Visual Working Memory Performance Based on Saccades in Children with and without Specific Learning Disorder: An Eye-Tracking Study
Some of the previous studies show that children with SLD have deficits in visual processing and working memory. Hence, the aim of this research was to investigate problems of visual working memory based on behavioral neuroscience method, using an eye tracker device. The method of present study was ex-post facto study. The participants included couple of twelve children with SLD (mean age=10.92)...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016